116 research outputs found
Semantically Informed Multiview Surface Refinement
We present a method to jointly refine the geometry and semantic segmentation
of 3D surface meshes. Our method alternates between updating the shape and the
semantic labels. In the geometry refinement step, the mesh is deformed with
variational energy minimization, such that it simultaneously maximizes
photo-consistency and the compatibility of the semantic segmentations across a
set of calibrated images. Label-specific shape priors account for interactions
between the geometry and the semantic labels in 3D. In the semantic
segmentation step, the labels on the mesh are updated with MRF inference, such
that they are compatible with the semantic segmentations in the input images.
Also, this step includes prior assumptions about the surface shape of different
semantic classes. The priors induce a tight coupling, where semantic
information influences the shape update and vice versa. Specifically, we
introduce priors that favor (i) adaptive smoothing, depending on the class
label; (ii) straightness of class boundaries; and (iii) semantic labels that
are consistent with the surface orientation. The novel mesh-based
reconstruction is evaluated in a series of experiments with real and synthetic
data. We compare both to state-of-the-art, voxel-based semantic 3D
reconstruction, and to purely geometric mesh refinement, and demonstrate that
the proposed scheme yields improved 3D geometry as well as an improved semantic
segmentation
Cataloging Public Objects Using Aerial and Street-Level Images – Urban Trees
Each corner of the inhabited world is imaged from multiple viewpoints with increasing frequency. Online map services like Google Maps or Here Maps provide direct access to huge amounts of densely sampled, georeferenced images from street view and aerial perspective. There is an opportunity to design computer vision systems that will help us search, catalog and monitor public infrastructure, buildings and artifacts. We explore the architecture and feasibility of such a system. The main technical challenge is combining test time information from multiple views of each geographic location (e.g., aerial and street views). We implement two modules: det2geo, which detects the set of locations of objects belonging to a given category, and geo2cat, which computes the fine-grained category of the object at a given location. We introduce a solution that adapts state-of-the-art CNN-based object detectors and classifiers. We test our method on “Pasadena Urban Trees”, a new dataset of 80,000 trees with geographic and species annotations, and show that combining multiple views significantly improves both tree detection and tree species classification, rivaling human performance
BiasBed -- Rigorous Texture Bias Evaluation
The well-documented presence of texture bias in modern convolutional neural
networks has led to a plethora of algorithms that promote an emphasis on shape
cues, often to support generalization to new domains. Yet, common datasets,
benchmarks and general model selection strategies are missing, and there is no
agreed, rigorous evaluation protocol. In this paper, we investigate
difficulties and limitations when training networks with reduced texture bias.
In particular, we also show that proper evaluation and meaningful comparisons
between methods are not trivial. We introduce BiasBed, a testbed for texture-
and style-biased training, including multiple datasets and a range of existing
algorithms. It comes with an extensive evaluation protocol that includes
rigorous hypothesis testing to gauge the significance of the results, despite
the considerable training instability of some style bias methods. Our extensive
experiments, shed new light on the need for careful, statistically founded
evaluation protocols for style bias (and beyond). E.g., we find that some
algorithms proposed in the literature do not significantly mitigate the impact
of style bias at all. With the release of BiasBed, we hope to foster a common
understanding of consistent and meaningful comparisons, and consequently faster
progress towards learning methods free of texture bias. Code is available at
https://github.com/D1noFuzi/BiasBe
Automated avalanche mapping from SPOT 6/7 satellite imagery with deep learning: results, evaluation, potential and limitations
Spatially dense and continuous information on avalanche occurrences is crucial for numerous safety-related applications such as avalanche warning, hazard zoning, hazard mitigation measures, forestry, risk management and numerical simulations. This information is today still collected in a non-systematic way by observers in the field. Current research has explored the application of remote sensing technology to fill this information gap by providing spatially continuous information on avalanche occurrences over large regions. Previous investigations have confirmed the high potential of avalanche mapping from remotely sensed imagery to complement existing databases. Currently, the bottleneck for fast data provision from optical data is the time-consuming manual mapping. In our study we deploy a slightly adapted DeepLabV3+, a state-of-the-art deep learning model, to automatically identify and map avalanches in SPOT 6/7 imagery from 24 January 2018 and 16 January 2019. We relied on 24 778 manually annotated avalanche polygons split into geographically disjointed regions for training, validating and testing. Additionally, we investigate generalization ability by testing our best model configuration on SPOT 6/7 data from 6 January 2018 and comparing it to avalanches we manually annotated for that purpose.
To assess the quality of the model results, we investigate the probability of detection (POD), the positive predictive value (PPV) and the F1 score. Additionally, we assessed the reproducibility of manually annotated avalanches in a small subset of our data. We achieved an average POD of 0.610, PPV of 0.668 and an F1 score of 0.625 in our test areas and found an F1 score in the same range for avalanche outlines annotated by different experts. Our model and approach are an important step towards a fast and comprehensive documentation of avalanche periods from optical satellite imagery in the future, complementing existing avalanche databases. This will have a large impact on safety-related applications, making mountain regions safer
Accuracy and Consistency of Space-based Vegetation Height Maps for Forest Dynamics in Alpine Terrain
Monitoring and understanding forest dynamics is essential for environmental
conservation and management. This is why the Swiss National Forest Inventory
(NFI) provides countrywide vegetation height maps at a spatial resolution of
0.5 m. Its long update time of 6 years, however, limits the temporal analysis
of forest dynamics. This can be improved by using spaceborne remote sensing and
deep learning to generate large-scale vegetation height maps in a
cost-effective way. In this paper, we present an in-depth analysis of these
methods for operational application in Switzerland. We generate annual,
countrywide vegetation height maps at a 10-meter ground sampling distance for
the years 2017 to 2020 based on Sentinel-2 satellite imagery. In comparison to
previous works, we conduct a large-scale and detailed stratified analysis
against a precise Airborne Laser Scanning reference dataset. This stratified
analysis reveals a close relationship between the model accuracy and the
topology, especially slope and aspect. We assess the potential of deep
learning-derived height maps for change detection and find that these maps can
indicate changes as small as 250 . Larger-scale changes caused by a winter
storm are detected with an F1-score of 0.77. Our results demonstrate that
vegetation height maps computed from satellite imagery with deep learning are a
valuable, complementary, cost-effective source of evidence to increase the
temporal resolution for national forest assessments
- …